Since the existing adaptive neighborhood based multi-view clustering algorithms do not consider the noise and the loss of consensus graph information, a Robust Multi-View Graph Clustering(RMVGC) algorithm based on adaptive neighborhood was proposed. Firstly, to avoid the influence of noise and outliers on the data, the Robust Principal Component Analysis(RPCA) model was used to learn multiple clean low-rank data from the original data. Secondly, the adaptive neighborhood learning was employed to directly fuse multiple clean low-rank data to obtain a clean consensus affinity graph, thus reducing the information loss in the process of graph fusion. Experimental results demonstrate that the Normalized Mutual Informations(NMI) of the proposed algorithm RMVGC is improved by 5.2, 1.36, 27.2, 4.66 and 5.85 percentage points, respectively, compared to the current popular multi-view clustering algorithms on MRSCV1, BBCSport, COIL20, ORL and UCI digits datasets. Meanwhile, in the proposed algorithm, the local structure of data is maintained, the robustness against the original data is enhanced, the quality of affinity graph is improved, and such that the proposed algorithm has great clustering performance on multi-view datasets.